Skip to content

feat: shared memory & drive store - #8

Merged
JTBroad merged 24 commits into
mainfrom
feat/memory-drive-store
Aug 1, 2026
Merged

feat: shared memory & drive store#8
JTBroad merged 24 commits into
mainfrom
feat/memory-drive-store

Conversation

@JTBroad

@JTBroad JTBroad commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Implements the memory & drive plan: a user-level Zettelkasten and an artifact
store that live outside any repo, so notes compound across projects and
generated files that shouldn't be committed have somewhere to go.

All twenty tasks (T0a–T19) are done.

Design

Both roots derive from stateDir, not the home directory. deriveServerPaths
already splits stateDir into baseDir/userdata or baseDir/dev, and that
split is what gives dev servers and the test suite isolated state. A hardcoded
~/.t3code default would bypass it — and since consolidation clears
daily.md, a test run would have destroyed real captured observations.

Scoping moves from storage to retrieval: one store, but every note and artifact
records where it came from, and recall filters by the current project. That is
what makes a shared store safe without giving up cross-project learning.

Surface

  • Migration 036 — four tables (drive_artifacts, memory_notes,
    memory_note_sources, memory_note_links) plus indexes. Deliberately not
    projection_*: projections replay from the event log, this is primary user
    state that nothing replays.
  • MCP toolkitmemory_append_daily, memory_read_daily, memory_search,
    drive_write_artifact, behind a new memory capability.
  • RPCsmemory.consolidate, readDaily, listNotes, getNote,
    listArtifacts, getArtifact.
  • UI — a workspace rail (Threads / Memory), a Memory workspace with Daily,
    Notes and Drive tabs, two settings rows, and a consolidate command in the
    palette.

Invariants worth protecting

  1. Dev and test runs never touch the real memory store.
  2. Provenance comes from the server and cannot be supplied by the model — no
    tool takes a projectSegment parameter, so a model cannot attribute an
    observation to another project or write into its drive.
  3. Concurrent captures never lose an entry — daily.md uses an O_APPEND write,
    not read-modify-write.
  4. A failed consolidation preserves the buffer. It rotates aside rather than
    clearing, and is discarded only after every note is written.
  5. Nothing reaches a prompt without a visible thread activity.

Two decisions to review

ContinuityBrief injection. There is no single prompt-composition point every
provider passes through — customInstructions is Copilot-only,
CodexDeveloperInstructions is Codex-only, Claude sends a preset system prompt,
and Cursor, Grok and OpenCode surface no hook at all. Injecting only where a
hook exists would make the same note change behaviour in some sessions and not
others with nothing on screen to explain it. So the brief is prepended to the
first user message, which behaves identically for all six.
BriefInjection.ts carries a TODO to revisit this as a provider-agnostic
session-preamble seam; that is provider-layer work.

drive_write_artifact was not in the task list. Nothing called
writeArtifact in production, so no turn could ever write an artifact and the
Drive tab would have been permanently empty. Added with the same anti-spoofing
shape as capture.

Verification

489 focused tests. Typecheck and lint clean on every changed package; the one
remaining apps/server error is pre-existing in HostPowerMonitor.ts,
confirmed on a clean tree.

Verified in the browser against isolated state: consolidation promotes entries
into notes with correctly differing project segments, clears the buffer, and is
visible live in the Daily tab; a hand-edited note file is picked up by the
self-healing reindex; provenance round-trips note → artifact → note; settings
persist, reset, and show resolved paths; the thread survives a Memory round trip
including in-progress composer text.

Three defects were found by that pass rather than by unit tests, all fixed here:
the sidebar's fixed left-0 panel drew over the rail, consolidation left the
open note stale, and the Threads button returned to the new-thread starter
instead of the open thread.

Not done

  • Promotion is mechanical — one entry becomes one note, first line becomes the
    title. No synthesis, no tag derivation, no Behavioral effect: line.
  • macOS window-control clearances were verified by simulating the insets and
    measuring, not in a real Electron build.
  • Nothing forgets yet; status is in the schema so demotion needs no migration.

Backing it out

Everything is additive. To disable without reverting, leave the memory
capability ungranted — the tools become unreachable and nothing writes. To
remove entirely, revert and drop the four tables; no existing table is altered
and no projection rebuild is affected.

🤖 Generated with Claude Code

JTBroad and others added 24 commits August 1, 2026 01:22
Adds `memoryDir` and `driveDir` to `ServerDerivedPaths`, created at
startup alongside the attachments directory.

Both stores are user-level and shared across every project, which is the
point -- a note about how the user works is not a per-repo fact. But they
are derived from `stateDir` rather than the home directory so the
existing dev/userdata split still applies. Consolidation clears the
capture buffer, so a home-directory default would let a dev server or a
test run destroy real captured notes. The regression test asserts dev and
production paths differ.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two optional roots on `ServerSettings`, following the
`addProjectBaseDirectory` pattern: a trimmed string that decodes to empty,
where empty means "use the stateDir-derived default". Keeping resolution
on the server rather than baking an absolute path into persisted state
means the default can move later without migrating anyone's settings.

Schema only -- resolution lives in apps/server, per this package's
schema-only rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root resolution (setting wins, stateDir default otherwise), a containment
guard, and stable project segments.

The guard is a port of `resolveAttachmentRelativePath` and keeps its
posture: a path that escapes its root returns null rather than being
clamped back inside, because a caller supplying a traversal is either a
bug or an attack and rewriting it hides both. Both roots are
user-configurable, which makes that guard more load-bearing here than it
is for attachments.

Project segments append a short digest of the absolute repository path to
the sanitized basename. Without it, two checkouts named `api` under
different parents would share one bucket and silently merge their notes.

`resolveProjectSegmentForThread` is deliberately not here: it needs a SQL
query against projection_threads/projection_projects, and belongs with the
code that has the client in scope rather than in a path module.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four tables: drive_artifacts, memory_notes, memory_note_sources, and
memory_note_links.

Deliberately not projection_* tables. Projections derive from the
orchestration event log and rebuild by replay; these hold primary,
user-owned state that nothing replays. Markdown files stay authoritative
for notes -- these rows are an index so recall and backlink queries do not
scan the corpus, and the consolidation reindex rebuilds them from
frontmatter.

Two index choices worth noting. The live-path index is unique but partial
(WHERE archived_at IS NULL): re-running the same task is the normal case,
so archiving a row must release its path for reuse rather than burning the
filename forever. And memory_note_links is indexed on to_note_id because
backlinks are the point of a Zettelkasten -- "which notes link here?"
has to be an indexed lookup, not a scan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two passes: known credential shapes (GitHub tokens and PATs, AWS key ids,
Slack tokens, JWTs, PEM private keys, credential-named assignments), then
a Shannon-entropy fallback for generated tokens no pattern names.

Redaction runs on write rather than read because the memory store is
shared across every project on the machine. A token captured while working
on one repository would otherwise sit in a file that every other project's
sessions read at session start; per-project stores would have contained
that, and this design deliberately does not.

Over-redaction is the failure mode that actually matters -- a redactor
that mangles commit hashes gets switched off, and then it protects
nothing -- so the entropy pass carries an allowlist and the tests assert
that SHAs, UUIDs, paths, URLs, emails, semver, and dotted identifiers all
survive untouched.

Two regression tests come from a real bug found while probing: the
original path allowlist matched [\w./-]+, which is also the shape of
base64, so every generated token passed through. It now keys on the path
separator and excludes "+"/"=" so base64 with slashes cannot smuggle past
the entropy check either.

Redaction is a mitigation, not a guarantee: a secret written in prose
still gets through, and the module says so.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`daily.md` is the short-term buffer every session appends observations to.
Each entry carries a provenance header written by the server -- capture
time, project segment, thread -- because consolidation needs to know which
project an observation came from to set a promoted note's scope, and a
model asked to supply that would eventually omit or invent it. An
unresolvable project records "unattributed" rather than failing the call:
losing an observation is worse than losing its attribution.

Appends are a single O_APPEND write, not read-modify-write. Verified by
temporarily reverting to read-modify-write: 18 of 20 concurrent captures
were lost. The concurrency test is the regression guard for that.

Bodies are redacted before reaching disk, and the raw body is never
logged or returned.

Adds `rotateDaily` alongside `clearDaily`. Truncating in place leaves a
window where an append landing between consolidation's read and its clear
is discarded unpromoted; renaming the buffer aside first means such an
append starts a fresh file and is picked up next cycle. The rotated file
is left on disk so a failed run can retry it. Consolidation should use
rotation -- `isReservedMemoryFile` keeps rotated buffers out of the note
reindex.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Permanent Zettelkasten notes as markdown files plus a SQL index.

The file is the source of truth: notes stay hand-editable and greppable,
so the index has to be reconstructible from them at any time. That is what
`reindexAll` is for, and why writes go file-first -- a file with no row is
repaired by the next reindex, while a row with no file is a dangling
reference nothing repairs. A malformed file is reported and skipped rather
than aborting the pass, so one bad note cannot block indexing the corpus.

Two things carry the long-term value and should survive refactoring.
Links record a `rel` and a `context` sentence, because a link that
explains itself is still useful a year later and a bare backlink is not.
And `backlinksFor` is an indexed lookup on to_note_id -- backlinks are the
point of a Zettelkasten, so answering "which notes link here?" must not
scan the corpus.

Reindex skips the daily buffer, rotated buffers, and the curated index
via `isReservedMemoryFile`. Indexing consolidation's own working files as
notes would be the cycle consuming its own output.

Note ordering puts the current project ahead of global notes, which is
what makes one shared store safe to read from inside a specific project.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generated files that should not be committed anywhere, made addressable:
a stable id, a content hash, and provenance back to the thread, turn, and
checkpoint that produced them. Observations say what happened; artifacts
say what was actually done, which is what makes them worth citing.

Files are namespaced under the project segment so they stay attributable
from the path alone, and every write goes through the containment guard.
A rejected path fails before touching disk or the database -- a partial
write with no row would leave an unreferenced file behind, tested
explicitly.

`archiveArtifact` is bookkeeping, not deletion: it releases the path via
the partial live-path index so a re-run can reuse a natural filename while
the previous output stays on disk.

`notesCiting` is the reverse of a note's sources. Provenance has to run
both ways or "why does the agent believe this?" has no clickable answer.

`artifactsCreatedSince` gives consolidation an explicitly bounded input
set rather than "everything in the directory".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Widens `McpCapability` to "preview" | "memory" and grants both when a
session credential is issued.

`requireMcpCapability` previously failed with
`PreviewAutomationUnavailableError`, whose `capability` field is the
literal "preview". A memory tool denial surfacing as a preview error would
be actively misleading in logs, so contracts gains
`McpCapabilityUnavailableError` for capabilities beyond preview. Preview
keeps its original error, so clients decoding it are unaffected.

The implementation covers every capability, so its inferred failure type
is the union of both errors -- which would force preview handlers to
declare a memory error they can never receive. Overloads narrow the
failure per capability instead.

`ThreadId` and `EnvironmentId` are imported from baseSchemas rather than
orchestration/environment: sourcing them elsewhere left them undefined at
module-init time and the schema failed to build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three tools -- memory_append_daily, memory_read_daily, memory_search --
registered beside the preview toolkit on the server's own MCP endpoint.
Every provider that speaks MCP gets them without a per-provider adapter,
which is what makes an observation captured in a Codex session
consolidatable by a Claude session and recallable in a Cursor one.

`memory_append_daily` takes only a body. Project, thread, and capture time
come from the MCP invocation scope, which the server issues when it mints
the credential, so a model cannot attribute an observation to a project it
is not working in -- doing so would quietly poison recall there. The
absence of those parameters is the guarantee, so a test asserts it.

Adds `resolveProjectForThread`, which was carved out of T3 because it
needs a SQL query and a path module should not own one. It keys on
projection_projects.workspace_root rather than the thread's worktree_path:
a worktree is per-branch, so keying on it would split one repository's
notes across every branch ever worked on.

Attribution is best-effort. A thread with no resolvable project records
"unattributed" rather than failing -- losing an observation is worse than
losing its attribution. Infrastructure faults die rather than surfacing as
tool errors, keeping the contract to "recorded" or "capability denied".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Promotes captured observations into permanent notes. The ordering is the
design, not incidental: lock, reindex, rotate the buffer aside, promote,
write a summary, and only then release the rotated buffer and advance the
marker. A failure anywhere leaves the rotated buffer on disk and the
marker unmoved, so the next run reconsiders the same entries rather than
losing them.

The lock is an exclusive file create ("wx"), so creation is the atomic
test, and it is released through Effect.ensuring -- a crash mid-run cannot
wedge it permanently. A second concurrent run reports "already-running"
rather than queueing or failing, and a test asserts the observation is
promoted exactly once.

Summaries go to a receipts/ subdirectory that the note reindex never
reads. A cycle that consumes its own output eventually spends its whole
budget reprocessing its exhaust, so a test runs two cycles and asserts the
first summary contributes no notes to the second.

Note ids are the capture timestamp plus a short digest of the body.
Position-based ids -- the first thing I wrote -- collide whenever two
observations share a capture second across different runs, and the
collision is silent: the second note overwrites the first. Hashing the
body also makes re-promoting an identical observation idempotent. The
regression test covers both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Assembles the budgeted grounding digest: 2,000 characters total with
per-section caps of identity 600, daily 500, brief 600, themes 300, notes
ranked current-project-first then by recency, and empty output when
nothing meaningful changed.

The constraints are the design, so the tests are adversarial: every
section oversized at once must still fit the total, each section must cap
independently, and a quiet period must produce an empty string rather than
a header with nothing under it. A digest that always fires trains the
model to ignore it.

Named ContinuityBrief, never "receipt" -- RuntimeReceiptBus already owns
that word for async runtime milestones.

INJECTION IS DEFERRED, deliberately. The task's first step was to confirm
a single prompt-composition point every provider passes through, and there
isn't one: customInstructions is Copilot-only and
CodexDeveloperInstructions is Codex-only. A brief injected for one
provider is worse than none, because behaviour would silently differ by
provider with nothing to indicate why. Wiring needs a per-provider
decision, so this module builds the text and nothing calls it yet. The
module docs say so.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A toolkit can be built correctly and still never be mounted, and no unit
test over the toolkit object catches that. This lists tools over HTTP
through the same registration layer McpHttpServer.layer composes, and
asserts all three memory tools appear alongside preview_status.

Verified to fail when MemoryToolkitRegistrationLive is removed from the
merge, so it is a real regression guard rather than a restatement of the
registration code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The brief is prepended to the first user message rather than composed
into a system prompt. No single prompt-composition point reaches every
provider -- customInstructions is Copilot-only, CodexDeveloperInstructions
is Codex-only, Claude sends a preset systemPrompt, and Cursor, Grok and
OpenCode surface no instruction hook at all. Wiring only the providers
that have a hook would make the same note change behaviour in some
sessions and not others with nothing on screen to explain why. Every
provider accepts messages, so the message path is the one seam that
behaves identically for all six.

BriefInjection.ts carries a TODO to revisit this as a provider-agnostic
session-preamble seam; that is provider-layer work, not memory work.

Recall runs on the opening turn only, is wrapped so any failure degrades
to no brief rather than blocking the turn, and is delimited so the model
reads it as context rather than as the user's request. Title and branch
generation continue to see the raw message text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t holds

`receipts/` collided with `RuntimeReceiptBus`, which already owns that
word for async runtime milestones. Two unrelated meanings for one term
is the confusion the plan called out; `summaries/` says what the
directory actually contains.

Mechanical: the reindex excludes the directory by filtering top-level
`.md` files, not by name, so nothing depended on the old string.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T12 and T13. Five endpoints: memory.consolidate plus listNotes,
getNote, listArtifacts and getArtifact.

Consolidation returns a tagged union so "already running" cannot be
mistaken for a failure -- it is the normal outcome of pressing the
button twice, and a client that models it as an error shows a red toast
for something that went fine. The tag also makes it impossible to read
the counts without handling the other cases first.

List endpoints carry a bounded limit in the schema rather than relying
on a server default, since adding a cap once clients depend on getting
everything is a breaking change. getNote returns backlinks alongside the
note so opening one is a single round trip instead of a waterfall.

Wire shapes are camelCase; the snake_case row shape is a storage detail
and leaking it would make every client field name a hostage to a future
migration. Absolute server paths (the consolidation summaryPath) are
dropped on the way out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T15. Both rows follow the "Add project starts in" pattern and touch all
four registration points: the row, changedSettingLabels, that memo's
dependency array, and restoreDefaults.

Two things this needed beyond the row markup:

ServerSettingsPatch is maintained by hand rather than derived from
ServerSettings, so both fields had to be added there too. Without that
the rows would have committed successfully and silently reverted on
reload -- covered now by a contracts test, since the next hand-added
setting has the same trap waiting.

Empty means "use the derived default", so the placeholder shows the
resolved path from a new optional memoryPaths field on the server config
snapshot. A blank input with no explanation reads as broken and invites
typing a path nobody needs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T16. The outcome-to-toast mapping lives in CommandPalette.logic.ts per
the repo's convention, so it is unit-tested without rendering.

"Already running" and "nothing to do" render as info, never as errors.
Pressing the button twice is the normal way to reach the first, and an
error toast for a successful no-op teaches people to distrust the
feature. A completed run that promoted nothing is also info rather than
success, since claiming success for zero work is just noise.

Consolidation goes through useConsolidateMemory so the palette entry and
the Memory workspace button (T18) share one in-flight state -- two
controls with independent running flags would let one look idle during a
run. The command is hidden rather than disabled while in flight: a
palette entry that does nothing when selected is worse than one that is
not offered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T17, T18 and T19, landed together so they share one verification pass.

The rail renders outside AppSidebarLayout, so Memory brings its own
sidebar and Threads keeps the thread sidebar untouched. Switching is
routing and nothing else -- no thread, panel, or selection store is
touched, which is what lets the active thread survive a round trip.
Filtering, sorting and selection live in MemoryView.logic.ts per the
repo's convention and are unit-tested without rendering.

T19's brief activity records every ContinuityBrief injection with the
exact injected text, so nothing reaches a prompt from the memory store
without a visible record. A failed activity is logged rather than
swallowed: it means an injection went unrecorded.

T19's other half needed a prerequisite that was missing from the task
list: nothing called writeArtifact in production, so no turn could ever
write an artifact, leaving the Drive tab permanently empty. Added
drive_write_artifact to the memory toolkit, with the same anti-spoofing
shape as capture -- the project bucket comes from the invocation scope,
never from a parameter, so a model cannot write into another project's
drive.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both found by the integrated verification pass, neither by a unit test.

The sidebar primitive positions its panel `fixed left-0`, which ignores
the rail's flex row and drew straight over it. The rail was present and
focusable in the accessibility tree the whole time, so only looking at
the page caught it. Offsetting the panel is scoped to the app shell
rather than changing the shared primitive for every other consumer.

Consolidating refreshed the note and artifact lists but not the open
detail pane, so a run left the selected note showing pre-run content --
and the note whose content just changed is exactly the one being read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rail renders flush to the top-left, which is where the desktop shell
draws close/minimize/zoom on macOS -- the first icon sat under them and
could not be clicked. The rail now offsets its top by the title bar
strip when those controls overlay the client area, and not in fullscreen
or in the browser, where there is nothing to clear.

Investigating that surfaced the same collision from a second source: the
sidebar's toggle is anchored to the window's left edge, so it was drawn
on top of the rail at x=12 regardless of platform. Window-edge insets
now add the rail width, and the one inset consumed as a margin *inside*
the sidebar takes it back off, so absolute positions are unchanged where
no rail is rendered.

That override is declared on the shell element rather than :root because
a custom property substitutes var() where it is declared -- on :root it
would always read the 0px default and silently do nothing.

The macOS detection AppSidebarLayout already had is now a shared hook, so
the rail and the sidebar cannot disagree about whether to inset.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Threads rail button pointed at a static "/", which is the index
route -- the new-thread starter. A round trip through Memory therefore
dropped whatever thread was open and landed on an empty composer.

The rail now records the last Threads-workspace route and returns there.
Memory routes are never recorded, or the button would point back into
Memory; neither are /pair and /connect, which render outside the app
shell and would drop the user into an auth screen they already cleared.
While already in Threads the button stays "/", matching what clicking
the active workspace does elsewhere.

The remembered path is module-scoped rather than React state because the
rail unmounts on those excluded routes, and a value that reset on
remount would lose exactly the thread this exists to preserve.

Still routing-only: no thread, panel, or selection store is read or
written. Verified against a seeded project that in-progress composer
text survives the round trip, not just the URL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The buffer was invisible in the app: Notes reads promoted notes, Drive
reads artifacts, and nothing surfaced where observations actually land
first. That is the worst thing to have hidden -- it is where over-eager
capture and redaction problems show up, and without it you only find out
at consolidation time, once the entries are already notes.

Tabs are now Daily, Notes, Drive: left to right is the lifecycle of an
observation. The workspace still opens on Notes, because Daily is empty
immediately after every consolidation and opening there would routinely
greet you with nothing.

memory.readDaily returns the raw file alongside server-parsed entries.
The raw text keeps redaction markers exactly as written; the parsed
entries save the client re-implementing the provenance header format,
which is a storage detail and would drift as a second parser.

Unattributed captures are counted separately rather than folded into the
total: one means thread resolution failed, so consolidation cannot scope
the note it produces. Consolidating refreshes the buffer first, since a
run clears it and a stale view would still list promoted entries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Memory workspace renders without AppSidebarLayout, and that layout is
what normally reserves the title bar strip -- so its tab bar started at
y=0 and the macOS traffic lights covered the left third of the Daily tab,
making it unclickable.

The clearance is applied at the shell, next to the branch that decides
whether a workspace gets AppSidebarLayout, rather than inside MemoryView.
The workspace should not have to know what the window chrome is doing,
and any future workspace rendered without the sidebar layout inherits the
fix instead of rediscovering the bug.

Measured with the macOS insets applied: rail icons at y=52 and y=92, tabs
at y=60, all clear of the y<=52 control strip. Browser mode is unchanged,
so no dead space appears where there are no window controls.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL labels Aug 1, 2026
@JTBroad
JTBroad merged commit 21ec9ce into main Aug 1, 2026
6 of 10 checks passed
@JTBroad
JTBroad deleted the feat/memory-drive-store branch August 1, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant